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mean time between failures 
cumulative test time 
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TECHNICAL PAPER 


AN OVERVIEW OF RELIABILITY GROWTH MODELS AND THEIR 
POTENTIAL USE FOR NASA APPLICATIONS 


I. SUMMARY 


This study provides an overview of reliability growth literature over the last 25 years. This 
includes a thorough literature review of the different areas of applications of reliability growth such as 
design, prediction, tracking/management, and demonstration. Various reliability growth models use a 
different basis on how they characterize growth. Different models are discussed in this report. 

Also, this report addresses the use of reliability growth models to NASA applications. This 
includes the application of these models to the space shuttle main engine (SSME) growth process. For 
potential NASA applications, we classify growth models in two groups: growth models for management 
and growth models for demonstration. Both groups of models are characterized in this report. 


II. INTRODUCTION 


While quality control has always been of some concern in manufacturing ventures, only during 
the last two to three decades has reliability become of major concern to all parts of the engineering 
community. This is because complex modern designs carry within them the possibility of various types 
of error and malfunction. 

Technology has been characterized in the past two decades by the development of complex 
systems containing a large number of subsystems, components, and parts. The trend to ever-larger and 
more complex systems is continuing with the development of space vehicles, weapons systems, 
communication systems, etc. Although the failure of a single inexpensive part or component may cause 
the failure of the entire system, reliability in general has not been considered as one of the system 
parameters such as cost or performance. It is very important to mention that unreliability has 
consequences in cost, idle time, psychological effects, and, in some cases, human life and national 
security. 

In the design and manufacturing of a complex system, such as the space transportation main 
engine (STME), the initial prototypes will invariably have significant reliability and performance 
deficiencies. Consequently, such a system is subjected to development testing. When a failure occurs 
during development testing, the cause is isolated and a corrective action is implemented. Reliability 
growth is a method for quantifying and monitoring system reliability during the development process 
through the collection and analysis of the relevant data. Although generally not contractually required, 
the existence of a reliability growth program during early development phases increases the likelihood 
that more problems will be resolved earlier in the program, thus reducing large costs later. 

Reliability growth is the positive improvement in reliability due to changes in product design or 
the manufacturing process. 1 It is expected that this process of finding problems and fixing them will 


result in increasing mean time between failures (MTBF). This has led to the development of various 
statistical models. These models characterize growth in different ways. Some use failure rate as a 
function of time, some use MTBF, and others are based on the idea of success probability. In fact, any 
reliability characterization that is appropriate may be used, as long as the model development will 
provide a tool to quantify growth. 

Any study of reliability growth has two main objectives. The first objective is the prediction of 
reliability at some future instant. In this case, the concern is to estimate rate of growth and to predict the 
achieved reliability up to the present time. This will help determine whether future reliability 
requirements will be met or not. In addition, the reliability growth study could provide information about 
growth patterns that might be useful to support decisions regarding the future of the program. Thus, one 
might investigate how growth of a system is related to such factors as number of tests, cost, design 
reviews, and number and types of design changes. For example, in the case of a liquid rocket engine, the 
reliability curve (reliability against number of tests) might “level off’ even when reliability is relatively 
low. That is, we find that continued testing and test-analyze-and-fix (TAAF) has reached “a point of 
diminishing return.” For all practical purposes, ultimate reliability of the hardware has been reached. If 
this value is not satisfactory, then a major design/process change might have to be introduced into the 
program. 

The second objective of any reliability growth study is statistical inference and estimation of 
reliability for demonstration purposes. In this case, the main concern is to verify a reliability requirement 
at a specified confidence level. To do this, data from both tracking phase (development) and certification 
phase are used. Specifically, to verify a certain reliability requirement, a test plan is developed to track 
reliability during development and to demonstrate the reliability requirement by incorporating the 
system-level testing during certification. 

Reliability managers have long been aware of the fact that the reliability of the system should 
improve as it progresses through development, but whether this growth will meet a targeted reliability is 
always a concern. To track reliability during development, a reliability growth management program can 
be used. This will furnish the manager with tools to evaluate and control program progress. A reliability 
growth management program will enable the manager to: 

1. Take advantage of experience gained in previous programs 

2. Evaluate different potential test plans and select the appropriate one 

3. Evaluate possible course of failures and the appropriate corrective actions when an ongoing 
program is experiencing problems 

4. Correctly evaluate the progress made by an ongoing program. 

A constant cause of concern for the development engineer and management is that reliability 
demands an excessive number of tests for reliability demonstration. For example, if a program 
requirement calls for 0.99-percent reliability with 90-percent confidence for demonstration, then 230 
tests with zero failure are required if a classical binomial model is used. Therefore, more innovative 
techniques need to be used. Reliability growth models are potential candidates for more efficient testing, 
which is the subject of this report. 
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in. OBJECTIVE 


The objective of this study is to explore the various reliability growth models for reliability 
tracking and the potential applicability of these models to NASA programs. 

To accomplish this task, the Redstone Scientific Information Center was used to compile a 
bibliography of 1 1 1 documents (books and articles) related to reliability growth. This literature review 
covers the past 25 years of work on reliability growth modeling, assessment, tracking, prediction, and 
control. In section V, a review of this literature is conducted, and models are then grouped into various 
categories for easy reference. Applications of reliability growth models are discussed in section VI. In 
section VII, some conclusions and recommendations are outlined. 


IV. NEED FOR GROWTH MODELS 


The development of a system often takes place by testing a system until it fails, then improving 
the design and testing again. At various times in this development process, it becomes important to 
assess and predict system reliability. The reliability growth of a system takes place due to changes 
introduced into the system structure. These changes make it difficult to estimate the system reliability 
using a classical model such as the binomial model. Also, classical models require the use of system test 
data for reliability demonstration. However, for some systems, test data are either not available or are 
available but very sparse. Thus, it is desirable to model the growth by a “growth model.” 

A reliability growth model is an analytical tool used to monitor the reliability progress during the 
developmental program, and to establish a test plan to demonstrate acceptable system reliability. The 
quantification provided by a growth model is valuable for proper management of a reliability program. 
Specifically, some of the advantages of using a reliability growth model are: 7 

1. Determining the intensity of TAAF to reach reliability objectives 

2. Predicting whether stated reliability objectives will be achieved 

3. Correlating reliability changes with reliability activities and tracking progress 

4. Planning for a reliability demonstration test 

5. Computing confidence limits to satisfy reliability requirements. 

At present, there is no lack of available models for the reliability engineers. The main problem is 
a lack of guidance for the selection of the best model suitable for individual application. Various 
reliability growth models use a different basis to characterize growth. For example, some are based on 
times between failures, some count the number of failures, and others use a homogeneous or 
nonhomogeneous Poisson process. The process of choosing a model is not a simple one. Balaban, 7 
Barr, 9 Gottfried, 52 Jayachandran, 60 and Jewell 61 have discussed model comparison. In some cases, this 
has aided in model selection. The information in this report is intended to help the reader in the selection 
and application of growth models. 
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V. OVERVIEW OF RELIABILITY GROWTH LITERATURE 


This section involves a literature review of the various types of reliability growth models. A 
listing of the papers and books from the literature search is included in the bibliography. In section V.A, 
the bibliography is categorized by subject areas for easy reference. Section V.B discusses selected 
models which use failure rate or times between failures as a criterion to characterize growth; while in 
section V.C, selected models using success probability are discussed. 


A. Bibliographical Characterization 

An extensive search of the literature was conducted to identify important reliability growth 
articles and books, especially those dealing with recent developments in reliability growth management, 
tracking, demonstration, and related applications. These articles and books are listed in the bibliography. 
Additional important references are cited in the papers and books listed. Important articles and books are 
then categorized according to subject area for easy reference. The following is the list of the categories 
established. 


Growth models 

Reference: 

1, 7, 9, 17, 20, 24, 33, 39, 52, 58, 60, 66, 83, 101, 104 

Tracking/management 

8, 10, 16, 26, 27, 28, 30, 54, 64, 69, 75, 99, 109, 1 1 1 

NASA applications 

50, 94, 95, 105 

Other applications 

18, 19, 24, 26, 64 

Reliability estimation 

3,25, 35, 36, 78,81,82, 107 

Case studies 

18, 19, 56 

Prediction 

14, 39, 40, 105 

Design reliability 

22, 38, 50, 88, 93, 94 

Parameter estimation/CI 

31,43, 47,68 

Simulation/Monte Carlo 

11,41,52, 108 

Bayesian analysis 

75,76, 85,91,99, 104, 106 

General theory 

4, 22, 29, 41, 51, 61, 62, 67, 71, 74, 79, 80, 84, 98, 101 
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B. MTBF Growth Models 


Based on the literature review, selected models which use MTBF as a basis to characterize 
growth are discussed in this section. It should be noted that the U.S. Army Material Systems Analysis 
Activity (AMSAA) model has been given more attention in the discussion because of the wide 
applicability, mathematical simplicity, and completeness of the model development. 

Duane’s Model : 

This model is one of the most widely used of all reliability growth models. Duane 39 used 
development data from several different systems, and concluded that the logarithm of observed 
cumulative mean time between failures (CMTBF) is a linear function of time. His model can be 
expressed as: 


N(t) = Kt l ~ a , (1) 

where AT is a constant, and a is the growth parameter. For this model, the cumulative failure rate A(r) is 
given by, 


A (t) = N(t)lt = Kr a . (2) 

Also, from equation (1), 

/i(r) = K(\-a)r a = (1 -a) - A (t) . (3) 

This shows the current failure rate p(f) is (1-a) times the cumulative failure rate, or talcing reciprocals, 
the current MTBF = 1/(1— or) times the CMTBF. 

Duane suggested that during the development phase, his model will hold. In practice, from the 
available data, one takes test hours and test failures, and plots them on a log-log plot. The constant K and 
the growth parameter a are determined from the graph. This is Duane’s postulate which is a 
deterministic learning curve formulation of reliability growth. 

Duane’s model predicts that the MTBF for the operational period following the completion of a 
development test of length T will be MTBF (7) = [KT a Y l . That is, the future MTBF is a function of the 
growth parameter and test length time T. A number of authors have suggested improvements on Duane’s 
model since its publication in 1964, pointing out the limitations of the model. In particular, Finkelstein 48 
claims that Duane’s model is unsuited for planning since its boundary conditions do not reflect 
development testing experience. These boundary conditions are the MTBF at times t = 0 and t = At t 
= 0, MTBF(O) = 0, and in the limit as it gets large, MTBF(°°) = Finkelstein modifies Duane’s model 
by introducing an additional parameter in the model. Gottfried 52 is also concerned about the behavior of 
Duane’s model at t - 0, and that the model implies continuous reliability growth in time, which is 
inconsistent with the basic premise of TAAF — that growth occurs only when problems are recognized 
and corrected. From equation (1) 


, = (^) 1/(1 ^ 


N(t)= 1,2,3,. 


(4) 
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By setting the mean time of occurrence of the first failure as follows: 


E(t)=o=$) m ~ a) ’ 

Gottfried examines reliability growth with the use of a variant of Duane’s model. This variant is based 
on stepwise growth which eliminates the above two problems with MTBF at t = 0 and t = «>. 

Duane’s model can be used for detecting the presence of reliability trends. Codier 24 describes the 
use of Duane’s growth curve to aid in determining the number of burn in cycles required to achieve a 
given failure rate for a radar LRU. For application of Duane’s model to the reliability growth evaluations 
of several aircraft test programs such as the F-16 and the B-52, see Bower. 19 Kasouf and Weiss 64 apply 
Duane’s model to an integrated missile reliability growth program. 

AMS A A Model : 

The AMSAA model was developed by Crow and is reported in appendix C of MIL HDBK 189. 1 
The AMSAA reliability growth model is designed for tracking the reliability within a test phase and not 
across tests phases. This model assumes that within a test phase, reliability growth can be modeled as a 
nonhomogeneous Poisson process (NHPP). This model, like Duane’s model, also assumes that within a 
test phase, the cumulative failure rate is linear on log-log-scale. However, the AMSAA model is a local 
(within test phase) model, while Duane’s model is a global (between phases) model. The AMSAA 
model evaluates the reliability growth that results from the introduction of design fixes into the system 
and not the reliability growth that may occur at the end of a test phase due to delayed fixes. For this 
model. Crow has developed rigorous statistical procedures useful for reliability growth tracking and 
demonstration. 

Let t denote the cumulative test time from the beginning of the test phase. Let o < y\ < y 2 
< y 3 ... < yx denote the cumulative test times of design modifications on the system. Let A; denote the 
constant failure rate during the ith time period (y,_i,y,) between modifications at y,-_ i and y,-. During 
development, more than one prototype is often tested. If the prototypes have the same basic 
configuration between corrections, then due to the constant failure rate assumption, the times y,- may be 
considered as cumulative test times on all prototypes up to ith correction. Also, on a cumulative time 
scale, Ni is the total number of failures for all systems during <y,_i,y,). 

Let N(t) be the total number of system failures by time t, then, 

(Ait if t is in the first interval, 

E\N(t)\ = < 1 

( A jyj+A 2 (t->’i) for t in the second interval, and so on. 

Thus, N(t) follows the NHPP 84 with mean value 6(t), 


e(t) = 



( 5 ) 


where p{x) is the intensity function given by, 


p(x) = A; for ig Cv'M,y ( ) , i = 1,2,... . 
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The AMSAA model assumes that the intensity function can be approximated by a continuous parametric 
function, i.e., 

p(t) = XfitP- 1 t>0,X>0,P>0 , (6) 

which is the Weibull failure rate function. Thus, the mean value function is: 

E(N(t)) = e(t) = XtP . (7) 

For (5 = 1, p{t) is constant, indicating an homogeneous Poisson process (HPP). For f$< 1, p{t) is 
decreasing, indicating reliability growth. For ft > 1, p{t) is increasing, indicating deterioration in system 
reliability. It should be noted that the model assumes that p{t) is approximated by a Weibull intensity 
function and not the Weibull distribution. Thus, statistical techniques used for the Weibull distribution 
are not applicable to p{t). 

The AMSAA model, like Duane’s model, can also be used for detecting the presence of 
reliability trends through the evaluation of the model parameters and the instantaneous failure rate. 
However, the AMSAA model gives an analytical solution which provides estimates at specified 
confidence levels, while Duane’s model provides a graphical solution which has limitations. 

To estimate the parameters of the AMSAA model, two procedures exist. One procedure is used 
for time terminated testing, and the second one is for failure terminated testing. 

Time Terminated Testing : 

Using the maximum likelihood estimation (MLE), the shape parameter P is estimated as follows: 

/j= ^ — • (8) 

N(nT- t,(nyi 

i = 1 

where N is the total number of failures up to test time T, and y,- are times of design modifications. 

Using the MLE estimator of /3, the estimator of A is given by 



Using the MLE of P and A gives the estimate of p{t), 

p(t) = X ptP~ l . (10) 

The reciprocal of p(t) provides an estimate of instantaneous MTBF. From equation (10), the 
estimate of p(t) can be written as: 


P(t) = jpPT^ = ^p 
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Expressions (8), (9), and (10) provide point estimates for (5, A, and MTBF. To establish a 
confidence interval for the MTBF, the following expression is used. 


Ln.z 

P(T) 


< MTBF < 


P(T) 1 


( 11 ) 


where z is the selected confidence level. T and Un, z are available in a specially provided table 
available in reference 1. In addition to point and interval estimation, reference 1 provides a procedure for 
testing the model fitness using the Cramer-Von Mises goodness-of-fit test. 


Failure Terminated Testing : 

In this case, the MLE of /J is given by, 

P N - f r l . 02 ) 

(N-l)(ny N - Z Vny, 

l - 1 

where N and y, are as defined in the time terminated testing case. The estimates of the scale parameter A 
and the intensity function are given as above. A special table to compute interval estimates of MTBF has 
been provided in this case also. Again, Cramer-Von Mises statistics can be used to test goodness-of-fit 
between the model and the observed data. Statistical precision and robustness of the AMSAA model 
estimators are discussed by Ziad. 111 * Wronka 109 applies the AMSAA model to grouped data to 
demonstrate reliability growth at an early development phase. 


The AMSAA reliability growth model is a fully developed model which has been widely used in 
various areas. To list a few applications: Crow 30 applies the AMSAA model for tracking reliability 
during the development phase of army programs. Benton and Crow 12 give an application to an 
integrated reliability growth program using different types of testing (such as early prototype, 
environmental, safety, TAAF, reliability demonstration, etc.). Ellis and Gibson 42 apply the AMSAA 
model to analyze repair times (the times required to complete maintenance actions). Safie 95 applies the 
AMSAA model to evaluate SSME reliability. It should be noted that this model, like Duane’s model, is 
sensitive to an early initial failure. 


IBM Model : 


This model is based on the solution of differential equations generated using the following two 
assumptions: 7 

1. Two types of failures are possible: random failures occurring at rate A (constant intensity 
function), and nonrandom failures due to design weakness defects. The number K of nonrandom failures 
is fixed but unknown at the beginning of testing. 

2. If M(t) is the number of nonrandom failures remaining at time t, the rate of change of M(t) is 

proportional to M(t). This implies: 


dM(t) 

dt 


= -K 2 M(t) 


(13) 
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and hence 


(14) 


M(t) = K x e~ K ^ K h K 2 > 0 , 


where K\ is the initial number of failures. 

An interesting feature of the IBM model is the ability of the model to predict the time when the 
system/equipment is “q” fraction debugged. In fact, from equation (14), 

q= , (15) 

thus, having estimated K 2 , say R 2 , we can find the time at which q = 0.90 of the nonrandom defects have 
been removed by solving equation (15). The solution is, in general, 

-in ( 0 . 10 ) 

* 0.90 “ g 

Equation (15) is a powerful tool because it can be used to determine the length of development 
testing if Ki has already been estimated. 

Considering both types of failures as defined in the two assumptions, the expected cumulative 
number of failures up to time t is given by V(t), 

V(t) = A t+K { -M(t) = A . (16) 


Using equation (16) 


Cum MTBF = t /{ A t+K x {\-e~ K *)) . 

An unattractive feature of the IBM model is that equation (16) is nonlinear. Thus, the estimation of A, 
K\, and Ki should be accomplished by iterative methods. 


Llovd and Lipow Model : 

This model, like the IBM model, is based on the solution of a differential equation. Let (pit) 
denote the mean time between failures. Lloyd and Lipow 71 propose a model in which the growth rate is 
inversely proportional to the square of time, i.e., 


dt 1 


^,>0 . 


Using equation (17), <f>(t) is given by 


<p(t) 


K r K x lt 

0 


t > K x !K 2 
0<t< K x !K 2 ’ 


(17) 


(18) 
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where K 2 is the constant of integration. As equation (18) indicated, MTBF is 0 for time period 
(0, K\!Ki ) and the limiting MTBF is K 2 . To estimate the growth parameter K\, let t'=l/t, 
then (pit') = K 2 -K l t'<p(t') is linear in t' with slope K\ and intercept K 2 . Both the parameters K\ and K 2 
can be estimated by the usual least-squares method. 

Aroef Model : 

Like the two previous models, this model is based on the solution of a differential equation. The 
Aroef model (reported in reference 7) considers that growth rate is directly proportional to the growth 
parameters and inversely proportional to t 2 , i.e., 

^l=K l (pm 2 , ( 19 ) 


solving 


(p (t) = K 2 e~ K ' l ‘ . (20) 

K 2 is the limiting MTBF since Yirn^ (p (t) = K 2 . Also ^lirn o 0(/) = 0, the initial MTBF, is 0. As in the 

previous model, the nonlinear form in equation (20) can be transformed to a simple linear equation and 
the least-squares method can be used to estimate the model parameters. 

ARINC Research Model : 

The ARINC research model proposed by Balaban 7 is described as 

<P(0,t) = KG(t) , (21) 

where K is the initial MTBF, (p(0,t) is the MTBF over the interval (0 ,t), and G(t) is the growth function 
defined as. 


G(t) = Af-(M-l) exp {-aft) . (22) 

As indicated in equations (21) and (22), this model has four parameters. Balaban suggests that K can be 
estimated by the first observed MTBF. Thus, in practice, only three parameters need to be estimated. As 
indicated in the following equation 


( n V n j = Qn a+ f3 Qnt , (23) 

the parameters of the model are determined by the following procedure: 

Step 1. Use the first applicable data point to estimate K. 

Step 2. For each observation period, calculate G(t ) = <P(Q),t)IK. 

Step 3. Choose a value of M (the maximum growth rate), then use the value of G(t) with the 
selected M to estimate a and /? using the least-squares method. 
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Step 4. Repeat step 3 for different values of M to improve the fit. 

Step 5. The best fit of step 4 yields least-squares estimates of a and ft from which the future 
growth function can be calculated as 


<5(t) = M-(M-l) exp (-atP) . 

To conclude this section, although only six models are discussed in section V.B, many other 
models using MTBF have been developed. Very few of these models have been widely used due to the 
difficulty involved in the estimation of the model parameters, the amount of subjectivity required to 
solve these models, and the lack of applicability of the model to real-life situations. It is important to 
emphasize that the AMSAA and the Duane’s models are the most commonly used models in reliability 
growth applications. 


C. Success Probability Models 

In this section, we discuss reliability growth models which use mission success probability as the 
basis to characterize growth. 

Llovd and Lipow Models : 

In reference 71, Lloyd and Lipow discuss a simple model for a system with one failure mode. 
Occasionally, a situation is experienced in a development program where a particular component 
repeatedly fails. If the system operates successfully for a given test, no corrective action is performed 
prior to the next test. However, if the system fails a corrective action is performed. Let a be the success 
probability of correcting the problem in the following test and p be the inherent probability of failure. 
Also, assume that any redesign effort either is completely successful or has no effect on the inherent 
failure probability p. 

Let p n ( 0) denote the probability that the failure mode has been eliminated before the nth test and 
p n ( l) denote the probability that the failure mode has not been eliminated before the next test, then 

p n ( i) = /vi(i)(i-p)+p n -ia)-pa-«) • ( 24 > 

Lloyd and Lipow assume that p\{\) = 1 and show that system reliability on the jlh test is 

Rj= \-{\-poc) H *p . 

However, if p\{\) = /I, where p< 1, then Rj can be expressed as 

Rj=l-A , (25) 

where the parameters A and C are given by 

A=(ip 

C =log( 1/(1 -/?«)) >0 . 

Methods for estimating parameters A and C are discussed in reference 71. 
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Lloyd and Lipow also consider a growth model in which they assume that a test program is 
conducted in N stages. At each stage, a certain number of tests are made and the number of failures 
recorded. The results of each stage are used to improve the items for the next stage. The authors use the 
growth model: 


Rj = Roo-a/j , ( 26 ) 

where Rj is the reliability during the j th stage of testing, and Roo is the maximum reliability as Roo 
and a are the model parameters. Let tij = sample size for the y'th test, sj = number of successes on 7 'th test, 

H = -j? X n i and c; = X , 

N j=l 1 ' j = lJ 

then initial estimates of Roo and a are obtained by solving: 


and 


a*- 


N(N+ 1 ) 


R^-a N , 


jj'Zs j = NR 00 -ccc l . 


If R„ and a represent estimates after a given iteration, the next pair of estimates are obtained from the 
following equations: 


where 



Dj = jf: (fc^-a/KX 1 -R^+a/K) . 


In reference 71, Lloyd and Lipow also give a lower confidence bound for Rj. 
Wolman Model : 


Wolman 110 considers a model with two types of failures: 

Type I: Inherent Failures 

Type II: Design Weakness Failures. 


He assumes that a number of type II failures are known, and, once a design weakness failure is 
eliminated, it will never again cause a system failure. He further assumes that the probability of a 
transient (system) failure may never increase due to corrective action, and the two types of failures occur 
independently. Let q 0 = P (inherent failure), and qi = P (design weakness failure, the system has 
experienced i failures which have been corrected), then 
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where q is the initial probability of design weakness failure, (3j is unknown, and (1 —fij ) represents a 
reduction in design failure probability by the y'th corrective action with (5 0 = 1. Let P„ = P (success after 
n failures are observed), then under this model 

P n = \-q 0 -q n _ x = 1 -q 0 -qp n ~ l if Pj = P (Wolman model) . (28) 

Wolman is interested in calculating quantities such as the probability of eliminating all K design 
weaknesses in K trials. It is important to mention that all parameters are assumed known, and, as a 
result, no estimations are involved. 

Barlow and Scheur Model : 

Barlow and Scheur 8 discuss a model for reliability growth assuming two types of failures as in 
the previous model. As in the Lloyd and Lipow test, the program is conducted in N stages. At each stage 
of experimentation, tests are run on similar items. The results of each stage of testing are used to 
improve items for further testing in the next stage. For the yth stage, cij, bj, and cy, which represent the 
number of inherent failures, the number of design weakness failures and the number of successes, 
respectively, are recorded. Let q 0 = P (inherent failure), qj — P (failure due to design weakness in the yth 
stage), and rj = P (of success in y'th stage), then rj, the reliability in y'th stage, is given by 

rj = \-qo-qj ■ < 29 ) 

Assuming that the sequence of q/s is nonincreasing, i.e., the increase in ry from stage to stage is 
accomplished by a decrease in qj which is the result of the appropriate corrective actions, the MLE of q 0 
and qj are given by 


q 0 = Z aj/jttej+bj+Cj) , 

(30) 

= (1 ~q 0 )b/(bj+Cj) j=l,2,...,N . 

(31) 


These are MLE if q x >q 2 t ... > q N . However, if qj < qj +i for somey (/ = 1,2,3,.. .,AM), then combine the 
y'th and the (j+ l)st stage and compute the MLE of <y/s using equation (31) for the (A/— 1) stages thus 
formed. This procedure is continued until the estimates of the q/s form a nonincreasing sequence. The 
MLE of the system reliability at the y'th stage is 


r i~ i-Qj—Qo • 

A lower confidence bound for rj is discussed in reference 8. 


( 32 ) 



Virene Model : 


Virene 105 considers the use of a Gompertz equation for reliability growth. His model has the 
following form: 


R, = ab ct 0<b< 1 , 0<c< 1 , 

where R t = reliability at stage t, a = upper limit approached by reliability as t -» and b and c are the 
other model parameters. A procedure to estimate these parameters is outlined in reference 105. Using the 
outlined procedure, Virene estimates a, b, and c by fitting observed data from the lunar orbiter spacecraft 
and Blue Scout launch vehicle. In his paper, Virene also provides a method to establish a lower 
confidence interval on reliability. 

Bonis Model : 

This model has the form 


where 


Rj — Roo~c[ aJ 1 , 


Rj = reliability on y'th test 
Roo = ultimate reliability 
q = initial reliability 

a = reliability growth factor, is the ratio of unreliability at the end of a stage to that at the end of 
the previous stage. 

Bonis 27 describes how his model can be used to determine reliability levels at each stage that will 
yield a final reliability consistent with an established goal. As test stage reliability data accumulate, the 
model is used to quantify actual progress, and this quantification of the data is necessary for proper 
reliability management. 

In conclusion, various reliability growth models which use success probability to characterize 
growth exist. Some of the models might have potential use for NASA applications such as the Lloyd and 
Lipow model and others as discussed. 


VI. APPLICATION OF GROWTH MODELS TO NASA PROGRAMS 


For NASA applications, the growth models can be classified into two groups: models for 
reliability management and models for reliability demonstration. The main difference between the two 
groups of models is that the one used for demonstration emphasizes the model accuracy and goodness of 
fit, while the other emphasizes analysis and tracking. While the SSME program is using growth models 
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for reliability demonstration, the STME program is planning to use reliability growth models for 
reliability management. This section provides an SSME example application of the reliability growth 
models for reliability demonstration purposes. 

Before going into the details of the SSME application, the following is a description of the 
process used to apply the AMSAA model to the SSME. Recalling from section V.B that the AMSAA 
model assumes that system failures follow an NHPP with Weibull intensity function given by. 


p{t) = XfltP- x , (33) 

where X is the scale parameter, and ft is the shape parameter. P is estimated by 

$= % — . < 34 > 

N (nT - jh (n xi 

i = i 

where N = number of failures, T = accumulated test time, and xi = failure times for failures i = 1,2,..., A. 
Using /?, the estimate of X is 

X = NIT P . (35) 

Using both parameter estimates, the instantaneous failure rate is given by 

m(7) = 1/p (7) , 

where 

p(T) = XpT ( P~ l) . (36) 

From equation (36), the lower confidence limit for MTBF is 

^<MTBF, (37) 

P(T) 

where L^y is the table value for N and y, the desired confidence level (reference 1, table Cl). 

The test of the null hypothesis that data fit the AMSAA model can be performed using the 
Cramer-Von Mises statistic cp. 


c 2 - J-+ y 

Cn ~ 1 2N + i = i 

where 

The null hypothesis is rejected if Cp > critical value. These critical values are available in reference 1 
for given n and a, the significance level. 

Applying the above procedure for SSME data (the SSME data are not up-to-date and are used for 
illustration purposes only), given the following SSME cumulative failure times, 92 


f jr i\P 2i-l 
,T) IN 


(38) 
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Failure (/') 

Cumulative Failure Times, 
xi, in seconds 

1 

505 

2 

10,348 

3 

10,872 

4 

15,516 

5 

15,844 

6 

48,168 

7 

48,476 

8 

55,606 

9 

78,724 

10 

97,648 

11 

158,674 

12 

206,712 

13 

270,242 


and the total time of T- 373,868 s. The P and X are determined using equations (34) and (35): 

P = 0.4228 and X = 0.05725 . 

Notice that a P of 0.4278 indicates a growth. Using these X and p, the instantaneous MTBF is 68,021. 
The MTBF is then used to calculate the engine reliability for a mission time of 520 s. 


R( 520) = g-mission duration/inst. MTBF _ Q 9924 


As described earlier, a lower bound on reliability can be established and testing the model goodness-of- 
fit can be performed using equations (37) and (38). 


VII. CONCLUSIONS 


In this report, we have provided a thorough review of the reliability growth literature with a 
complete bibliography of the materials searched. Also, we have identified and discussed some selected 
models such as the AMSAA, Duane, IBM, ARINC Research, Lloyd and Lipow, Barlow and Scheur, 
etc., models. Some of these models have been used for NASA applications. Other models might have 
potential use in future NASA programs. 

As a concluding remark, it is important to keep in mind that using reliability growth during 
development and certification will provide a tool to track and improve reliability, enhance the 
effectiveness of a test program, and evaluate the effectiveness of design process changes. 
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